See through near-eye display

ABSTRACT

The various embodiments include a near-eye display having a transmissive display and a diffractive micro-lens array. The transmissive display may be positioned relative to the diffractive micro-lens array so that the distance between the transmissive display and the diffractive micro-lens array is be approximately equal to focal length of the diffractive micro-lens array. The transmissive display may also be positioned relative to the diffractive micro-lens array so that a percentage of light emitted from the transmissive display is diffracted by the micro-lens array and collimated into focus on a retina of a human eye. The transmissive display may be further positioned relative to the diffractive micro-lens array so that light from a real world scene passes through transparent portions of the transmissive display and is diffracted by the micro-lens array out of focus of the human eye.

FIELD OF THE INVENTION

The present application relates generally to displays for computingdevices, and more specifically to a near-eye display that allows a humaneye to simultaneously focus on real world images and computer generatedimages that can be overlaid with the real world images.

BACKGROUND

Cellular and wireless communication technologies have seen explosivegrowth over the past several years. Cellular service providers now offera wide array of features and services that provide their users withunprecedented levels of access to information, resources andcommunications. To keep pace with these service enhancements, mobileelectronic devices (e.g., cellular phones, tablets, laptops, etc.) havebecome more feature rich, and now commonly include powerful processors,graphics hardware, cameras, global positioning system (GPS) receivers,and many other components for connecting users to friends, work, leisureactivities and entertainment. Due to these improvements, mobile deviceusers can now execute powerful software applications on their mobiledevices, such as augmented reality software applications that combinereal world images from a user's physical environment withcomputer-generated imagery. As a result of these and other enhancements,mobile devices have become ubiquitous and mobile device users now expectto have access to content, data and communications at any time, in anyplace.

With the ubiquity of mobile devices, and the nearly continuous access toapplications and communications that they provide, mobile device usersare being drawn into a deeper engagement with their mobile devices andbecoming less aware of their physical surroundings. For these and otherreasons, an electronic display that enables mobile device users tosimultaneously focus on their physical surroundings and computergenerated images/content will be beneficial to consumers.

SUMMARY OF THE INVENTION

An embodiment near-eye display may include a transmissive electronicdisplay, and a diffractive micro-lens array configured to diffract apercentage of incoming light to form a virtual image of the display at adistance greater than or equal to 250 mm from a user's eye, in which thedistance between the transmissive electronic display and the micro-lensarray is about the focal length of the micro-lens array. The diffractivemicro-lens array may be partially diffractive and partially transparent.The transmissive electronic display may include a plurality of pixelsand the diffractive micro-lens array may be positioned relative to thepixels so that about fifty percent of light emitted from each pixel isdiffracted into focus on a retina of a user's eye. The transmissiveelectronic display may include a plurality of transparent portions andthe diffractive micro-lens array may be configured and positionedrelative to the transmissive electronic display so that light from adistant scene passes unaltered through the transparent portions of thetransmissive display. The diffractive micro-lens array may be positionedrelative to the transmissive electronic display so that about fiftypercent of the light from the real world scene passes through thediffractive micro-lens array and forms an image on the retina. Thediffractive micro-lens array may be positioned relative to thetransmissive electronic display so that about fifty percent of the lightfrom the real world scene is diffracted by the micro-lens array out offocus of the user's retina. The diffractive micro-lens array may befabricated on a glass substrate and/or fabricated into a photopolymerfilm as a volume hologram. The diffractive micro-lens array andtransmissive electronic display may be fabricated into an optical lens,which may be part of or attached to a pair of eyeglasses. Thetransmissive electronic display may a liquid crystal display. Thetransmissive electronic display may be an organic light emitting diodedisplay, which may be a transparent organic light emitting diodedisplay.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and constitutepart of this specification, illustrate exemplary embodiments of theinvention, and together with the general description given above and thedetailed description given below, serve to explain the features of theinvention.

FIGS. 1A-1C are illustrations of components commonly included in priorart augmented reality glasses.

FIG. 2 is an illustration of imaging system suitable for displayingelectronically generated images on a prior art heads-up display system.

FIG. 3A is an illustration of an embodiment near-eye display system inthe form of eyeglasses.

FIG. 3B is an illustration of an embodiment near-eye display systemsuitable for use in eyeglasses.

FIGS. 4-7 are ray tracing diagrams that illustrate light paths in anembodiment near-eye display configured to focus light from a real-worldscene and light generated from an electronic display on a human retina.

FIG. 8 is an illustration of a holographic micro-lens array having adiffraction pattern suitable for use in an embodiment near-eye display.

FIG. 9 is an illustration of an embodiment near-eye display systemhaving a holographic micro-lens array configured to form an image inclose proximity to a human eye so that the formed image appears at adistance from the human eye.

FIG. 10 is a block diagram illustrating example components of anembodiment head mounted display (HMD) system.

FIG. 11 is a block diagram of a mobile computing device suitable for usewith the various embodiments.

DETAILED DESCRIPTION

The various embodiments will be described in detail with reference tothe accompanying drawings. Wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or like parts.References made to particular examples and implementations are forillustrative purposes, and are not intended to limit the scope of theinvention or the claims.

The word “exemplary” is used herein to mean “serving as an example,instance, or illustration.” Any implementation described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other implementations.

The terms “mobile device,” “user equipment,” and “hand-held device” areused interchangeably herein to refer to any one or all of cellulartelephones, smartphones, personal or mobile multi-media players,personal data assistants (PDA's), laptop computers, tablet computers,smartbooks, ultrabooks, palm-top computers, wireless electronic mailreceivers, multimedia Internet enabled cellular telephones, wirelessgaming controllers, and similar personal electronic devices whichinclude a programmable processor, memory, and circuitry for sendingand/or receiving wireless communication signals.

The term “pixel” is used herein to refer to smallest addressablediscrete element in an electronic display device. Typically, the greaterthe number of pixels per unit area, the greater the resolution of theelectronic display device.

The phrase “heads-up display” and its acronym “HUD” are used herein torefer to any electronic display system that presents the user withinformation without requiring users to look away from their usualviewpoints.

The term “near-eye display” is used herein to refer to an opticaldisplay device that may be worn in close proximity to one or both of auser's eyes. A near-eye display may be included in a contact lens,eyeglasses, head mounted displays (e.g., as part of a helmet or on theface of an individual), heads-up display, virtual reality glasses,augmented reality glasses, electronic goggles, and other similartechnologies/devices.

Due to recent advances in mobile device technologies, mobile deviceusers can now execute powerful software applications on their mobiledevices, such as augmented reality software applications that combinereal world images from a user's physical environment withcomputer-generated imagery. An augmented reality application may addgraphics, sounds, and/or haptic feedback to the natural world thatsurrounds a user of the application. Information about people and/orobjects present in the user's physical environment may be retrieved froma database presented to the user on an electronic display so that theuser can view and/or interact with the representations of the real-worldpeople/objects. As an example, a mobile device augmented realityapplication may capture an image of a building in the mobile deviceuser's field of view, perform image matching or other operations toidentify the building, retrieve information pertaining to the identifiedbuilding from a database, and overlay the retrieved information on theimage of the building so that the user can view the information withinthe context of the building. As another example, a mobile deviceaugmented reality application may identify the presence of a human facein the user's vicinity, perform facial recognition operations toidentify the individual whose face was detected, retrieve an avatar, awebsite, or information associated with the identified individual from adatabase (e.g., a local database, Internet, etc.), and display theretrieved information in proximity to the detected human face. Whilethese new mobile device features, capabilities, and applications (e.g.,augmented reality applications) may be beneficial to consumers, theyalso have the potential to draw users into deeper engagement with theirmobile devices, distract them from their physical surroundings, and/orisolate them from the real-world.

The various embodiments provide a near-eye display capable of displayingelectronic or computer generated images on eyeglasses so they appearsuperimposed on the real-world scene (i.e., what the user would seewithout the glasses). This enables the user to view the generated imagein the context of the real-world scene without requiring the user tolook away from his/her usual viewpoints. In an embodiment, a near-eyedisplay may be embedded in an optical lens of a pair of lightweight andinconspicuous eyeglasses or contact lenses that may be worn very closeto a human eye for extended periods of time without causing significanteye fatigue or blocking the user's peripheral vision. Embedding thenear-eye display in the glasses eliminates the bulk and weight ofprojection and long-focal length displays used in conventional near-eyedisplays and heads up displays.

In an embodiment, the near-eye display may include a transmissivedisplay and a diffractive micro-lens array. The transmissive display maybe positioned very close to the diffractive micro-lens array so that thedistance between the transmissive display and the diffractive micro-lensarray is be approximately equal to the focal length of the diffractivemicro-lens array. In an embodiment, the transmissive display may bepositioned relative to the diffractive micro-lens array so that apercentage of light emitted from the transmissive display is diffractedby the micro-lens array and collimated so that, when worn, the displayimages will focus on the retina of a user's eye so that the lightappears to originate at a distance of about 250 mm or more from theuser. Also, the transmissive display may be positioned relative to thediffractive micro-lens array so that light from a real world scenepasses through transparent portions of the transmissive display withoutdiffraction by the micro-lens array so that it can be seen by the user.Any light from the real world scene that is diffracted by the micro-lensarray will be rendered out of focus on the human eye such that thediffracted light is ignored by the user's brain.

Generally, the human eye functions by focusing light rays through thelens onto an assemblage of photoreceptor cells in the human retina.Focusing of light is achieved by contracting or relaxing a series ofmuscles that change the physical shape of the eye and thus the distancebetween the lens and the retina. To focus on nearby objects, a normalhuman eye contracts various muscles to cause the eye lens to bulge andreduce the distance between the lens and the retina. The closestdistance that human eye can focus is from 10 cm (young people) to 50 cm(old people). When the muscles are relaxed, the eye elongates and theeye is “focused at infinity.” A human eye is typically focused atinfinity when the light publication distance (i.e., distance between thegenerated image and the user's eye) is around two feet. The averagehuman eye is much more comfortable when focused at infinity than whenfocused on nearby objects, and prolonged focus on nearby objectstypically causes eye fatigue. For these and other reasons, conventionaldisplay technologies and image generation techniques are not suitablefor use in near-eye displays, such as eye glasses, which are worn inclose proximity to the eye (e.g., less than a few inches).

FIGS. 1A-1C illustrate various components of a pair of prior artaugmented reality glasses 100 configured to generate an electronic imagein close proximity to the human eye. FIG. 1A illustrates that augmentedreality glasses 100 may include a frame 102, two optical lenses 106, aprocessor 114, a memory 116 and a projector 108. The projector 108 maybe embedded in arm portions 104 of the frame 102 and configured toproject images onto the optical lenses 106.

FIG. 1B illustrates example components of a typical projector 108 thatmay be embedded an arm portion 104 of the augmented reality glasses 100.The projector 108 may include a communication circuitry 130 for sendingand/or receiving information to and from the processor 114, alight-emitting diode (LED) module 132, a light tunnel 134, ahomogenizing lens 136, an optical display 138, a fold mirror 140, andother well known components commonly included in prior art projectors.Briefly, the LED module 132 generates a beam of light that travelsthrough the light tunnel 134 and the homogenizing lens 136 before beingdeflected towards the optical display 138. The optical display 138combines the received light beam with other light beams (e.g., othercolors) to generate an image. The generated image may be reflected offthe fold mirror 140, through a collimator and/or a relay imaging system,and onto the optical lens 106 of the augmented reality glasses 100.

Due to the relatively short distances between the optical lens 106 andthe eye lens of the user, a relay imaging system must extend thedistance between the generated image and the user's eye (i.e., lightpublication distance) to a distance larger than 50 cm so that the usersof all ages can focus on the generated image.

FIG. 1C illustrates components of a relay imaging system 160 suitablefor displaying an image on an optical lens 106 of prior art augmentedreality glasses 100. The relay imaging system 160 extends the distancebetween the generated image and the user's eye so that the user canfocus on the generated image. In the example illustrated in FIG. 1C, therelay imaging system 160 includes a composite optical lens 162 having afirst lens 164, a second lens 166, and a waveguide 168. The waveguide168 includes a plurality of reflective sections 170 and a transparentsection 172. Light beams from the optical display 138 enter a compositelens 162 via a collimator 174 that narrows and focuses the light beamsonto reflective sections 170 of the waveguide 168. The light beamsbounce off the reflective sections 170 until they reach the transparentsection 172 of the waveguide 168. The light beams pass through thetransparent section 172, through the first lens 164, onto the eye lens180, and is focused on the retina 182 of a user of the augmented realityglasses 100.

As mentioned above, a human eye is typically focused at infinity whenthe light publication distance is around two feet. Achieving thisdistance using relay optics often requires bulky lenses that may causethe glasses to be uncomfortable to wear, conspicuous, unattractive,heavy, and otherwise unappealing to consumers. Further, relay opticstypically block a significant amount of incoming light from a real-worldscene, which may reduce the user's peripheral vision and/or isolate theuser from his/her natural environment. In addition, by blockingsignificant amounts of incoming light from a real-world scene, existingaugmented reality solutions do not provide a seamless integrationbetween the generated image and a real-world scene.

Another known technology for integrating projected images with realworld scenes is a heads up display. FIG. 2 illustrates an exampleimaging system 200 for displaying electronically generated images on aconventional heads-up display configured to present the user withinformation without requiring the user to look away from his or herusual viewpoints. FIG. 2 illustrates that light 202 emitted from theprojector 108 may be reflected off one or more mirrored orsemi-transparent portions 204 of an optical lens or windshield 206 ontothe eye lens 180 of the user/user. Light 210 from a real word scene maypass through the non-mirrored portions of the optical lens/windshield206 to reach the user's eye lens 180.

When the imaging system 200 is implemented in a conventional heads-updisplay system, such as those used in automobiles and aircraft, theimage displayed on the projector 108 must be located at a significantdistance (e.g., two or more feet) from the user's eyes so that the useris able to focus on the generated image (e.g., the user's eye may remainfocused at infinity). Therefore, existing heads-up display solutions andtechnologies are not suitable for implementation in near-eye displays,such as eyeglasses and contact lenses, which are worn in close proximityto the user's eye.

In the case of augmented reality glasses, projecting the generate imagedirectly on a glass surface of an optical lens will not form an image onthe retina, and use of re-imaging lenses will cause the real-world sceneto become distorted by the lens. If the generated image is projected tothe side of the glasses, light 210 from the real world scene and light202 from the projector do not overlap on the user's retina. Further, theimage from the projector may not be in the field of view of the user'seye when his/her eyes are focused on the real world scene, and viceversa. For these and other reasons, implementing the imaging system 200in near-eye display does not provide a seamless integration between thereal-world scene and the generated images.

In addition to the above-mentioned limitation of existing solutions, therelay imaging systems discussed above require that the projector 108 bepositioned inside or within the optical lens/windshield 162, 206 so thatlight 202 emitted from the projector 108 can be reflected off thereflective portions 170, 204 of the optical lens/windshield 162, 206. Inthe case of augmented reality glasses, this is typically achieved bypositioning the projector 180 within the frame 102, which requires thatthe frame 102 be made of thick, heavy, and/or bulky material thatconsumers may find uncomfortable, conspicuous, unattractive, heavy, andotherwise unappealing. For these and other reasons, existingvirtual/augmented reality systems and near-eye display solutions are notsuitable for use in near-eye displays and/or are unappealing toconsumers.

The various embodiments provide a near-eye display capable of combininga real-world scene (i.e., what the user would see without the display)with an electronic or computer generated image so that the user viewsthe generated image in the context of the real-world scene withoutcausing significant eye fatigue or blocking peripheral vision. Thevarious embodiments provide a near-eye display system that allows ahuman eye to simultaneously focus on both the real world and generatedimages by relaying the generated image to infinity without the bulkyrelay optics required in prior art solutions. Various embodimentsseamlessly integrate the real-world scene and the generated images in amanner that does not contribute to eye fatigue or cause userdistraction/isolation.

FIG. 3A illustrates components of an embodiment near-eye display system300 in the form of a pair of eyeglasses 306. The eyeglasses 306 may berimless, wireframe, plastic frame, wood frame, horn-rimmed, browline,pince-nez, safety glasses, sunglasses, or any other type of eyewear oreyeglasses currently available or which may be developed in the future.In the example illustrated in FIG. 3A, the eyeglasses 306 include aframe 306 for supporting optical lenses 304 in front of a user's eyes.The frame 306 may be constructed from a lightweight material (e.g.,polymer, alloy, metal, wood, etc.) suitable for supporting the opticallenses 304 in front of the user's eyes. The optical lenses 304 may beprescription lenses, non-prescription lenses, vision correction lenses,magnification lenses, polarized lenses, darkened lenses, photochromiclenses, or just a piece of transparent glass or plastic substrate, orany other type of eyeglass lens currently available or which may bedeveloped in the future.

The optical lens 304 may include a near-eye display 302. The near-eyedisplay 302 may include a transmissive display 308 and a diffractivemicro-lens array 310. In an embodiment, the near-eye display 302 may beembedded in or attached to the optical lens 304 so that incoming lightfrom a real world scene passes through the transmissive display 308 andthe micro-lens array 310 before reaching the user's eye lens 180.

In various embodiments, the near-eye display 302 may include twosubstrate layers 314 and/or a spacer 312 positioned between thetransmissive display 308 and the diffractive micro-lens array 310. Thesubstrate layers 314 may be glass, plastic, or any other suitabletransparent or semi-transparent substrate known in the art. The spacer312 may be a solid spacer (e.g., glass, plastic, etc.), a liquid spacer(e.g., liquid crystals, etc.), or a gas spacer (e.g., air, titaniumoxide, etc.).

In various embodiments, the near-eye display 302 may be fabricated sothat the spacer 312 has a thickness of two hundred (200) micrometers,one hundred (100) micrometers, ten (10) micrometers, etc. In anembodiment, the near-eye display 302 may be fabricated to have a totalthickness of about one millimeter or less. In an embodiment, thenear-eye display 302 may be fabricated so that the distance between thetransmissive display 308 and the diffractive micro-lens array 310 isapproximately equal to the focal length of the diffractive micro-lensarray 310. In an embodiment, the near-eye display 302 may be fabricatedso that each mircolens in the micro-lens array 310 is approximately thesame size as a pixel.

The micro-lens array 310 may be a partially diffractive and partiallytransparent diffractive optical element (DOE) having a phase profile anda diffraction efficiency. In an embodiment, the micro-lens array 310 maybe fabricated to have a diffraction efficiency of about 50%. In anembodiment, the micro-lens array 310 may be fabricated so thatapproximately 50% of the light from the real-world scene is not affectedby the micro-lens and the other approximately 50% light is focused toform a virtual image of the transmissive display 308 at a distance ofabout 250 mm or more from the user's eye lens. In an embodiment, thediffraction efficiency of the micro-lens array 310 may be proportionalto the phase profile of the micro-lens array 310.

The transmissive display 308 may be any electronic visual display thatis transparent or semi-transparent and/or which uses ambient light as anillumination source. The transmissive display 308 may include, or makeuse of, any or all of a variety of commercially available displays ordisplay technologies, including technologies relating to liquid crystaldisplays (LCDs), light-emitting diodes (LEDs), organic light-emittingdiode (OLEDs), surface-conduction electron-emitter displays (SEDs),electroluminescence (EL) displays, photoluminescence displays, plasmadisplay panels (PDPs), field emission displays (FEDs), nanotube fieldeffect displays, micro-mirror displays, micoelectromechanical (MEMs)displays, electrochromic displays, electrophoretic displays and/or othersimilar display technologies currently available or which may bedeveloped in the future.

In an embodiment, the transmissive display 308 may be a transmissive orpartially transmissive liquid crystal display. Briefly, a liquid crystaldisplay (LCD) is an electronic display that uses light modulatingproperties of liquid crystals whose reflectance and/or transmittance oflight change when an electric field is applied. Liquid crystals may bearranged to form pixels within a transparent or semi-transparent liquidcrystal display device. Liquid crystal displays generally do not producelight, and need illumination from ambient light or a light source toproduce a visible image. In an embodiment, the transmissive display 308may be a liquid crystal display that produces a visible image fromambient light.

FIG. 3B illustrates components of an embodiment near-eye display system350 suitable for inclusion in a pair of eyeglasses. The near-eye displaysystem 350 may include a transmissive display 308, a diffractivemicro-lens array 310, two substrate layers 314, and a spacer 312. Thetransmissive display 308 may be illuminated by a front light illuminator352 that includes light sources (e.g., LEDs) coupled into the substrate314 from the side of the substrate. The light may propagate inside thesubstrate 314 through total internal reflection, and may be reflectedtowards the LCD through small reflectors distributed on one of thesubstrate 314 surfaces. The distribution of the small reflectors may bedesigned to provide a uniform and efficient illumination on the LCD.

In various embodiments, the transmissive display 308 may be a liquidcrystal display that does not include a front light, includes a partialfront light, or includes a front light that illuminates the liquidcrystal display from the front substrate.

In an embodiment, the transmissive display 308 may be an electronicdisplay that uses light-emitting diodes (LEDs) as an illuminationsource, such as to produce a visible image on a liquid crystal display.In embodiment, the transmissive display 308 may be partiallytransmissive LED display.

A light-emitting diode is semiconductor light source that may be used toproduce a visible image and/or as a front light on display devices. Alight-emitting diode typically includes a semiconducting material dopedwith impurities to create a p-n junction. Electrons and holes (i.e.,charge-carriers) flow into the p-n junction from the electrodes (i.e.,the anode and the cathode) when a voltage or current is applied to thelight-emitting diode. When an electron meets a hole, it releases energythrough the emission of a photon, producing visible light.

In an embodiment, the transmissive display 308 may be an organiclight-emitting diode (OLED) display. In conventional light-emittingdiodes, the semiconducting material is typically formed from a varietyof inorganic materials (e.g., InGaN, GaP, GaN, phosphor, etc.). Anorganic light-emitting diode (OLED) is a light-emitting diode in whichan organic semiconducting material is situated between the twoelectrodes (i.e., the anode and the cathode), all of which may bedisposed on a substrate (e.g., glass, plastic, foil, etc.). OLEDdisplays do not require a backlight, may be fully transparent orsemi-transparent when not producing a visible image (e.g., when turnedoff), and do not consume a significant amount of power. The OLEDs may beprinted of a variety of substrates, and may be used in conjunction with,or independent of, liquid crystal display technologies.

For ease of reference, throughout this application, liquid crystaldisplay (LCD) is used as an exemplary technology used by thetransmissive display 408. However, it should be noted that the use ofLCD terminology in this application is only for purposes ofillustration, and should not be construed to limit the scope of theclaims to a particular technology unless expressly recited by theclaims.

FIGS. 4-7 illustrate light paths in an embodiment near-eye display 400configured to simultaneously focus light from a real-world scene andlight generated from an electronic display on a human retina. In theexamples illustrated in FIGS. 4-7, the near-eye display 400 includes atransmissive display 308, a spacer 312, and a diffractive micro-lensarray 310. The micro-lens array 310 may be partially diffractive andpartially transparent diffractive optical element (DOE) fabricated todiffract a percentage (e.g., about 50%, etc.) of incoming light to thelensing ray path.

In an embodiment, the near-eye display 302 may be fabricated so that thedistance between the transmissive display 308 and the diffractivemicro-lens array 310 is approximately equal to focal length of thediffractive micro-lens array 310. In an embodiment, the near-eye display400 may be fabricated so that approximately fifty (50) percent of lightemitted from each pixel (or group of pixels) 410 is diffracted andcollimated into focus on the retina 182. In an embodiment, the near-eyedisplay 302 may be fabricated so that the percentage of light focused onthe retina 182 is proportional to the diffraction efficiency of themicro-lens array 310.

FIG. 4 illustrates that light 412 from the real world scene passesunaltered through the transparent portions of the transmissive display308 to the micro-lens array 310. A first percentage (e.g., 50%) of light412 from the real world scene that passes unaltered through thetransparent portions of the transmissive display 308 is diffracted bythe micro-lens array 310 and a second percentage (e.g., 50%) of thelight 412 from the real world scene is not diffracted by the micro-lensarray 310. The near-eye display 400 may be fabricated so that thediffracted light 416 is out of focus on the retina and disregarded asscatter light or background noise by the human retina 182. Light 412from the real world scene that is not diffracted by the micro-lens array310 (i.e., non-diffracted light 406) is focused by the eye lens on theretina 182 in normal fashion.

When an LCD is used as the transmissive display 308 and ambient light isused as the illumination source, the off state pixel may be transparentwhile the on state pixel may block or partially block the ambient lightpassing through the display. In this arrangement, most of the LCD pixelsthat do not display information are transparent and allow the real worldscene to be seen by the wearer, while a small amount of pixels displayimage (e.g., text, etc.) appear to be merged in the real world scene toprovide information about the scene (e.g., map direction, etc.).

FIG. 5 illustrates that when an illumination source (e.g., LED, etc.) ofthe transmissive display 308 is powered and/or the pixels 410 are turnedon, light is emitted from an LED or reduced light through LCD travelsthrough the pixels 410 in the transmissive display 308 to the micro-lensarray 310. A first percentage (e.g., about 50%) of light 408 emittedfrom the transmissive display 308 is diffracted by the micro-lens array310 (i.e., diffracted light 404) and a second percentage (e.g., about50%) of the light 408 emitted from the transmissive display 308 is notdiffracted by the micro-lens array 310 (i.e., non-diffracted light 405).The near-eye display 400 may be fabricated so that the diffracted light404 is collimated and subsequently focused on the retina 182 by the eyelens 180, and the non-diffracted light 405 passes through, out of focus,so that it may be disregarded as scatter light or background noise bythe human retina 182. This may be achieved by fabricating the near-eyedisplay 302 so that the distance between the transmissive display 308and the diffractive micro-lens array 310 is approximately equal to focallength of the diffractive micro-lens array 310.

FIG. 6 illustrates that non-diffracted light 406 from the real worldscene and diffracted light 404 from the transmissive display 308 may besimultaneously focused on the human retina 182. As discussed above,light 412 from the real world scene that passes unaltered through thetransparent portions of the transmissive display 308 and is notdiffracted by the micro-lens array 310 (i.e., non-diffracted light 406)is naturally collimated because the light source is at infinity; it issubsequently focused on the retina 182 by the eye lens 180 in the normalfashion. As discussed above, the near-eye display 400 may be fabricatedso that the light 408 emitted from the transmissive display 308 anddiffracted by the micro-lens array 310 (i.e., diffracted light 404) iscollimated by the micro-lens 310 and subsequently focused on the retina182 by the eye lens 180. In an embodiment, the near-eye display 400 maybe fabricated so that the light 408 emitted from the transmissivedisplay 308 and diffracted by the micro-lens array 310 is focused on theretina 182 in a manner that enables the display pixels to form a virtualimage at a distance of about 250 mm or more from the eye.

FIG. 7 illustrates that diffracted light 416 from the real world sceneand non-diffracted light 405 from the transmissive display 308 are notfocused on the retina 182 and/or are disregarded by the human brain asscatter light or background noise. That is, due to the distance betweenthe transmissive display 308 and the diffractive micro-lens array 310(e.g., approximately equal to focal length of the diffractive micro-lensarray 310), the diffracted light 416 is diffracted out of focus so thatit may be disregarded by the retina as background noise. Likewise, thedue to the close proximity of the transmissive display 308 to the eyelens 180, the non-diffracted light 405 originating from the transmissivedisplay 308 is also out of focus and/or disregarded by the retina 182 asbackground noise.

In the various embodiment, the diffraction efficiency of the micro-lensarray 310 may controlled by fabricating the micro-lens array 310 to havea specific phase profile and/or a plurality of deviations having variousdepths, the frequency and sizes of which may control the percentage oflight diffracted and/or focused on the retina 182. In an embodiment, themicro-lens array 410 may be shaped so that the other ˜50% light from thedisplay pixel is nearly collimated in a manner that enables the displaypixels to form a virtual image at a distance of about 250 mm or morefrom the eye.

In an embodiment, the micro-lens array 310 may be fabricated on a glasssubstrate using lithographic methods, similar to those used in siliconchip manufacturing. In this embodiment, very small lenses may be formedin a glass substrate by selective etching using photo lithographyfollowed by subtractive processes, such as etching, to transfer a phaseprofile onto glass surface. Photolithographic techniques may be used toform the tiny lens structure of the micro-lens array 310. The lensstructure may include small lenses that cover approximately 50% of thesurface area of each lens, while the rest of the surface area may beformed to enable light to pass through the lens un-diffracted.

In an embodiment, the micro structure of the micro-lens array fabricatedby photolithographic technique may cover approximately 100% of thesurface area of each lens, while the diffraction efficiency of the eachlens is about 50%. In this arrangement, 50% of the incoming light willbe diffracted and 50% of the incoming light will not be diffracted. Assuch, 50% of the light from the display pixels and 50% of the light fromthe real world scene will be focused on the retina simultaneously.

In an embodiment, the micro-lenses and/or micro-lens array 310 may befabricated using a holographic method. The micro-lenses and/ormicro-lens array 310 may include volume holographic micro-lensesfabricated to focus the electronic image on the retina while eliminatingimage cross-talk between the adjacent lenses. For example, the volumeholographic micro-lenses may be fabricated so that only light from thepixels associated with their companion lens element in the micromicro-lens array are diffracted, and so that light from neighboringpixels do not satisfy the Bragg condition and will not be diffracted.

In an embodiment, the micro-lens array 310 may be a holographicmicro-lens array fabricated via a holographic recording method throughthe coherent interference of two waves. In an embodiment, theholographic micro-lens array 310 may be fabricated by recording ahologram on a holographic medium, for example, a photopolymer film togenerate an optic diffractive optical element. The diffractionefficiency of the holographic micro-lens array 310 may be controlled byvarying the exposure time of the photopolymer to achieve a preciserefractive index modulation. The holographic micro-lens may befabricated on a photopolymer film that may be applied to the back of thetransmissive display 308. In an embodiment, the photopolymer film may bebetween about 10 micrometers and about 30 micrometers in thickness. Inan embodiment, the photopolymer film may be less than about 10micrometers in thickness.

FIG. 8 is an illustration of recording holographic micro-lens array 802having a diffraction pattern suitable for use in an embodiment near-eyedisplay. The holographic micro-lens array 802 may be fabricated byexposing a photographic film 804 with light from a laser that followstwo paths, one path emanating from points positioned where atransmissive display will be relative to the photographic film 804, andthe other path from a source (e.g., real world scene) more than 250 mmin distance from the holographic micro-lens array. The holographicmicro-lens array 802 may be fabricated so that light waves from thesetwo sources (i.e., transmissive display and source) form a diffractionpattern, which when recorded in the photographic film, generates aholographic optical element in the form of the holographic micro-lensarray 802.

In an embodiment, the micro-lens array 310 may be fabricated with aholographic printing technique. A holographic printer is able tocalculate the interference fringe patterns correspond to theinterference of two waves depicted in FIG. 8 and writes the fringes witha focused beam (or beams) into a photosensitive material that willchange the refractive index after exposure to light.

FIG. 9 illustrates that when light from a display pixel 410 located atthe focus or focal point of the recording wave near the hologram 802strikes the photographic film 802, the holographic micro-lens array 802may form a virtual image of the pixel 410 so that the image appears tobe greater than 250 mm from the eye lens 180 and/or so that the eye mayview the image while focused at infinity.

One of the advantages of including a volume hologram as the micro-lensarray is its ability to eliminate pixel cross-talk, because light fromneighboring pixel will not be able to reconstruct the virtual image wavedue to Bragg mismatch. A diffractive optical lens may be associated withchromatic aberration that will affect the image quality on retina. Sinceaberration is linearly proportional to lens focal length (with a fixed Fnumber), when lens size is small, the aberration can be significantlyreduced.

In various embodiments, the holographic micro-lens array 802 may be adiffractive optical lens having a narrow band response, which mayeliminate chromatic aberration. In an embodiment, the micro-lens arraymay be fabricated to include a sufficiently narrow band diffractive lensthat light from a narrow spectrum band “sees” the hologram and isdiffracted to form a sharp image on retina, whereas light outside of thespectrum band will not “see” the hologram and will be unaffected. Suchdiffractive lens can be made as a volume hologram.

In various embodiments, the near-eye display may be included as part ofa head mounted display (HMD) system (e.g., helmet, eyeglasses, etc.),which may include a processor, a memory, a display and/or a camera in asingle device (e.g., eyeglasses) or may be configured to operate as anaccessory to a mobile device processor (e.g., the processor of a cellphone, tablet computer, smartphone, etc.).

FIG. 10 illustrates example components of an embodiment head mounteddisplay (HMD) system 1000. The head mounted display system 1000 includeseyeglasses 1010 configured to operate as an accessory to a mobile device1020. The eyeglasses 1010 may include an optical lens 1002 having anear-eye display 302, and a wireless radio 1004 and/or a wirelessinterface and/or circuitry for sending and/or receiving information toand from the mobile device 1020. The near-eye display system 1000 mayalso include one or more sensors 1006, such as cameras, microphones, eyetracking components, accelerometers, gyroscopes, magnetic sensors,optical sensors, mechanical or electronic level sensors, inertialsensors, electronic compasses, and other known sensors that may beincluded in modern mobile electronic devices. Information collected fromthe sensors 1006 may be transmitted to the mobile device 1020 coupled tothe eyeglasses 1010 via the wireless radio 1004.

In various embodiments, the sensors 1006 may include one or more sensorsfor scanning or collecting information (e.g., light, location ofobjects, etc.) from the user's environment (e.g., room, etc.), distancemeasuring sensors (e.g., a laser or sonic range finder) configured tomeasure distances to various objects present in the user's environment,sensors for detecting user inputs, sensors configured to collectioninformation regarding the up/down/level orientation of the near-eyedisplay 302 (e.g., by sensing the gravity force orientation), the user'shead position/orientation (and from that viewing perspective), and/orregarding left/right orientation and movement. In an embodiment, thesensors 1006 may include one or more sensors configured to detect userinputs, such as spoken voice commands, gestures (e.g., hand movements),eye movements, and other forms of inputs which when recognized by themobile device 1020, may cause that device to execute a specific orcorresponding command or operation.

In an embodiment, the sensors 1006 may include an eye tracking componentconfigured to track a location of the user's eye relative to thenear-eye display 302. The eye tracking component may communicate withthe mobile device 1020 so that the mobile device processor 1012 maygenerate images for display on near-eye display 302 relative to theposition of the user's eye and/or based on real-world images present inthe users line of view. The eye tracking component may also detect eyemovements (blinks, left motion, right motion, up motion, down motion,etc.) as a source of user input, and communication the detected userinputs to the mobile device 1020. In an embodiment, the eye trackingcomponent may be configured to obtain an image of the user's eye anddetermine the location of the pupil within the eye socket.

In an embodiment, the sensors 1006 may include a microphone forcapturing verbal user inputs or commands, which may be communicated tothe mobile device 1020 via the wireless radio 1004. The processor 1012may receive audio signals from the microphone and process the receivedaudio signals using speech recognition processes/techniques. In anembodiment, the processor 1012 may be configured to compare receivedaudio signals to audio patterns of one or more commands stored in amemory in order to recognize a spoken command. For example, theprocessor 1012 may be configured to monitor audio inputs for a fewpredetermined command words. The processor 1012 may be configured toapply a detection algorithm to the received audio so that it onlyresponds to particular predefined audio commands, or commands proceededby a predefined attention command (e.g., “computer” or “execute” etc.).The processor 1012 may be configured to recognize these spoken words asa command input, and implement corresponding actions to update theimages displayed on the near-eye display 302.

In an embodiment, the main processing of the head mounted display system1000 may be performed on a processor 1012 of the mobile device 1020. Inan embodiment, the processor 1012 may be configured to perform variousimage processing and data analysis operations, such as analyzing imagescaptured by the sensors 1006 (e.g., a camera) to estimate distances toobjects (e.g., via trigonometric analysis of stereo images), performfacial recognition operations, identify logos, perform keyword orpicture searches, etc.

In an embodiment, the processor 1012 may be configured to generate avirtual object for display on the near-eye display 302. The processor1012 may be configured to calculate display-relevant parameters,including distance and orientation with respect to the sensors 1006 thatcorrespond to a display location of the virtual object. The virtualobject may be any virtual object, including, for example, text,graphics, images and 3D shapes. When presented on the near-eye display302, the virtual object may be positioned at/on designated locationswithin the surrounding environment to create the experience of augmentedreality and/or enable user interactions with the virtual object. Thesensors 1006 may enable natural interactions with the virtual objectsand digital assets (e.g., documents, pictures, videos, etc.) via gesturecontrols, touch manipulations, highlighting portions of the virtualobject, etc. Recognizable gestures may be stored or organized in theform of a gesture dictionary that stores movement data or patterns forrecognizing gestures, including pokes, pats, taps, pushes, guiding,flicks, turning, rotating, grabbing and pulling, two hands with palmsopen for panning images, drawing (e.g., finger painting), forming shapeswith fingers (e.g., an “OK” sign), and swipes, all of which may beaccomplished on, in close proximity to, or addressing the direction of(in relation to the user) the apparent location of a virtual object in agenerated display.

FIG. 11 is a system block diagram of a mobile device suitable for usewith any of the embodiments. A typical mobile device 1100 may include aprocessor 1101 coupled to internal memory 1102, a display 1103, and to aspeaker 1154. Additionally, the mobile device 1100 may include anantenna 1104 for sending and receiving electromagnetic radiation thatmay be connected to a wireless data link and/or cellular telephonetransceiver 1105 coupled to the processor 1101 and a mobile multimediabroadcast receiver 1106 coupled to the processor 1101. Mobile devices1100 typically also include menu selection buttons or rocker switches1108 for receiving user inputs.

The processor 1101 may be any programmable microprocessor, microcomputeror multiple processor chip or chips that can be configured by softwareinstructions (applications) to perform a variety of functions, includingthe functions of the various embodiments described above. In somedevices, multiple processors 1101 may be provided, such as one processordedicated to wireless communication functions and one processordedicated to running other applications. Typically, softwareapplications may be stored in the internal memory 1102 before they areaccessed and loaded into the processor 1101. The processor 1101 mayinclude internal memory sufficient to store the application softwareinstructions. In many devices the internal memory may be a volatile ornonvolatile memory, such as flash memory, or a mixture of both. For thepurposes of this description, a general reference to memory refers tomemory accessible by the processor 1101 including internal memory orremovable memory plugged into the device and memory within the processor1101 itself The foregoing method descriptions and the process flowdiagrams are provided merely as illustrative examples and are notintended to require or imply that the steps of the various embodimentsmust be performed in the order presented. As will be appreciated by oneof skill in the art the order of steps in the foregoing embodiments maybe performed in any order. Words such as “thereafter,” “then,” “next,”etc. are not intended to limit the order of the steps; these words aresimply used to guide the reader through the description of the methods.Further, any reference to claim elements in the singular, for example,using the articles “a,” “an” or “the” is not to be construed as limitingthe element to the singular.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the embodiments disclosedherein may be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and steps have been described above generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, but such implementation decisions should not beinterpreted as causing a departure from the scope of the presentinvention.

The hardware used to implement the various illustrative logics, logicalblocks, modules, and circuits described in connection with theembodiments disclosed herein may be implemented or performed with ageneral purpose processor, a digital signal processor (DSP), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA) or other programmable logic device, discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. Ageneral-purpose processor may be a microprocessor, but, in thealternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration. Alternatively, some steps or methods may be performed bycircuitry that is specific to a given function.

In one or more exemplary aspects, the functions described may beimplemented in hardware, software, firmware, or any combination thereof.If implemented in software, the functions may be stored as one or moreinstructions or code on a non-transitory computer-readable medium ornon-transitory processor-readable medium. The steps of a method oralgorithm disclosed herein may be embodied in a processor-executablesoftware module which may reside on a non-transitory computer-readableor processor-readable storage medium. Non-transitory computer-readableor processor-readable storage media may be any storage media that may beaccessed by a computer or a processor. By way of example but notlimitation, such non-transitory computer-readable or processor-readablemedia may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium that may be used to store desired programcode in the form of instructions or data structures and that may beaccessed by a computer. Disk and disc, as used herein, includes compactdisc (CD), laser disc, optical disc, digital versatile disc (DVD),floppy disk, and blu-ray disc where disks usually reproduce datamagnetically, while discs reproduce data optically with lasers.Combinations of the above are also included within the scope ofnon-transitory computer-readable and processor-readable media.Additionally, the operations of a method or algorithm may reside as oneor any combination or set of codes and/or instructions on anon-transitory processor-readable medium and/or computer-readablemedium, which may be incorporated into a computer program product.

The preceding description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, the present invention is notintended to be limited to the embodiments shown herein but is to beaccorded the widest scope consistent with the following claims and theprinciples and novel features disclosed herein.

What is claimed is:
 1. A near-eye display for wearing by a user,comprising: a transmissive electronic display configured to emit light;and a diffractive micro-lens array positioned between the transmissiveelectronic display and an eye of the user such that light directedtoward the user passes through the diffractive micro-lens array beforereaching the eye of the user, the light directed toward the usercomprising light from a real world scene and light emitted from thetransmissive electronic display, wherein: the diffractive micro-lensarray diffracts a first percentage of the light emitted from thetransmissive electronic display, and the diffractive micro-lens arrayallows a second percentage of the light emitted from the transmissiveelectronic display to pass therethrough not diffracted, the secondpercentage of the light emitted from the transmissive electronic displayis out of focus at a point of a retina of the eye, the first percentageof the light emitted from the transmissive electronic display is focusedat the point of the retina to form a virtual image that appears to theuser to originate at a distance that is greater than or equal to 250 mmfrom the eye, and the distance between the transmissive electronicdisplay and the diffractive micro-lens array is about a focal length ofthe diffractive micro-lens array.
 2. The near-eye display of claim 1,wherein the diffractive micro-lens array is partially diffractive andpartially transparent.
 3. The near-eye display of claim 1, wherein: thetransmissive electronic display includes a plurality of pixels; and thediffractive micro-lens array is positioned relative to the plurality ofpixels so that about fifty percent of light emitted from each pixel isdiffracted into focus on the point of the retina of the eye when focusedat infinity.
 4. The near-eye display of claim 1, wherein: thetransmissive electronic display includes a plurality of transparentportions through which the light from the real world scene passesunaltered; and the diffractive micro-lens array is positioned relativeto the transmissive electronic display to receive the light from thereal world scene through the plurality of transparent portions.
 5. Thenear-eye display of claim 4, wherein the diffractive micro-lens array ispositioned relative to the transmissive electronic display so that aboutfifty percent of the light from the real world scene that passesunaltered through the plurality of transparent portions of thetransmissive electronic display passes through the diffractivemicro-lens array not diffracted and is focused at a second point on theretina to form a second an image on the retina of the eye.
 6. Thenear-eye display of claim 5, wherein the diffractive micro-lens array ispositioned relative to the transmissive electronic display so that aboutfifty percent of the light from the real world scene that passesunaltered through the plurality of transparent portions of thetransmissive electronic display is diffracted by the diffractivemicro-lens array out of focus of the retina of the eye when focused atinfinity.
 7. The near-eye display of claim 1, wherein the diffractivemicro-lens array is fabricated on a glass substrate.
 8. The near-eyedisplay of claim 1, wherein the diffractive micro-lens array isfabricated into a photopolymer film as a volume hologram.
 9. Thenear-eye display of claim 1, wherein the diffractive micro-lens arrayand the transmissive electronic display are fabricated into an opticallens.
 10. The near-eye display of claim 1, wherein the transmissiveelectronic display is a liquid crystal display.
 11. The near-eye displayof claim 1, wherein the transmissive electronic display is an organiclight emitting diode display.
 12. The near-eye display of claim 11,wherein the organic light emitting diode display is a transparentorganic light emitting diode display.
 13. The near-eye display of claim1, wherein the transmissive electronic display is attached to a pair ofeyeglasses.
 14. A near-eye display for wearing by a user, comprising:means for emitting light; and means for receiving light directed towardthe user, the light directed toward the user comprising light from areal world scene and light emitted from the means for emitting light;the means for receiving the light directed toward the user beingpositioned between the means for emitting light and an eye of the usersuch that the light directed toward the user passes through the meansfor receiving the light directed toward the user before reaching a pointof a retina in the eye of the user; the means for receiving the lightdirected toward the user includes: means for diffracting a firstpercentage of the light emitted from the means for emitting light, andmeans for allowing a second percentage of the light emitted from themeans for emitting light to pass therethrough not diffracted; the secondpercentage of the light emitted from the means for emitting light is outof focus at a point of a retina of the eye; the first percentage of thelight emitted from the means for emitting light is focused at the pointof the retina to form a virtual image that appears to the user tooriginate at a distance greater than or equal to 250 mm from the eye;and the means for receiving the light directed toward the user beingpositioned relative to the means for emitting light such that thedistance between the means for receiving the light directed toward theuser and the means for emitting light is about a focal length of themeans for receiving the light directed toward the user.
 15. The near-eyedisplay of claim 14, wherein the means for receiving the light directedtoward the user comprises: means for partially diffracting light from acomputer generated image; and means for partially transmitting the lightfrom the real world scene.
 16. The near-eye display of claim 14,wherein: the means for emitting light comprises means for generating aplurality of pixels; and the means for diffracting the first percentageof the light emitted from the means for emitting light comprises meansfor diffracting fifty percent of light emitted from each pixel intofocus on the point of the retina of the eye when focused at infinity.17. The near-eye display of claim 14, wherein: the means for emittinglight comprises a plurality of transparent portions through which thelight from the real world scene passes unaltered; and the means forreceiving the light directed toward the user comprises means forreceiving the light from the real world scene through the transparentportions.